IT equipment cooling

ABSTRACT

A system for cooling gas heated by passing the gas over heat-producing equipment to cool the equipment comprises a heat exchanger including a first heat transfer mechanism configured to transfer heat from the heated gas to a first coolant, and a first condensing module connected for fluid communication with the heat exchanger and including second and third heat transfer mechanisms, the first condensing module being configured to transfer heat through the second and third heat transfer mechanisms from the first coolant to second and third coolants in the second and third heat transfer mechanisms, respectively.

RELATED APPLICATIONS

This reissue application claims priority to U.S. Pat. No. 7,418,825which was a divisional application claims of and claimed priority toU.S. application Ser. No. 10/993,329, filed Nov. 19, 2004, now U.S. Pat.No. 7,165,412, each of which is incorporated by reference in itsentirety.

BACKGROUND

Communications and information technology equipment is commonly designedfor mounting to racks and for housing within enclosures (often includedin the term “rack”). Equipment racks are used to contain and to arrangecommunications and information technology equipment, such as servers,CPUs, internetworking equipment and storage devices, in small wiringclosets as well as equipment rooms and large data centers. An equipmentrack can be an open configuration and can be housed within a rackenclosure, although the enclosure may be included when referring to arack. A standard rack typically includes front-mounting rails to whichmultiple units of equipment, such as servers and CPUs, are mounted andstacked vertically within the rack. The equipment capacity of a standardrack relates to the height of the mounting rails. The height is set at astandard increment of 1.75 inches, which is expressed as “U” units orthe “U” height capacity of a rack. A typical U height or value of a rackis 42 U, and an exemplary industry standard rack is about six tosix-and-a-half feet high, by about 24 inches wide, by about 40 inchesdeep. A standard rack at any given time can be sparsely or denselypopulated with a variety of different components as well as withcomponents from different manufacturers.

Most rack-mounted communications and information technology equipmentconsumes electrical power and generates heat. Heat produced byrack-mounted equipment can have adverse effects on the performance,reliability and useful life of the equipment components. In particular,rack-mounted equipment housed within an enclosure is particularlyvulnerable to heat build-up and hot spots produced within the confinesof the enclosure during operation. The amount of heat generated by arack of equipment is dependent on the amount of electrical power drawnby equipment in the rack during operation. Heat output of a rack canvary from a few watts per U of rack capacity up to about 950 watts per U(with this upper end continuing to rise), depending on the number andthe type of components mounted to the rack. Also, users ofcommunications and information technology equipment add, remove, andrearrange rack-mounted components as their needs change and new needsdevelop. The amount of heat a given rack or enclosure can generate,therefore, can vary considerably from a few tens of watts up to about40,000 watts, and this upper end continues to increase.

Rack-mounted equipment typically cools itself by drawing air along afront side or air inlet side of a rack, drawing air through itscomponents, and subsequently exhausting air from a rear or vent side ofthe rack. Airflow requirements to provide sufficient air for cooling canvary considerably as a result of different numbers and types ofrack-mounted components and different configurations of racks andenclosures.

Equipment rooms and data centers are typically equipped with an airconditioning or cooling system that supplies and circulates cool air toracks. Many air conditioning or cooling systems, such as the systemdisclosed in U.S. Pat. No. 6,494,050, use an equipment room or datacenter that has a raised floor construction to facilitate airconditioning and circulation functions. These systems typically use openfloor tiles and floor grills or vents to deliver cool air from an airpassageway disposed below the raised floor of an equipment room. Openfloor tiles and floor grills or vents are typically located in front ofequipment racks, and along aisles between rows of racks arrangedside-by-side.

The cooling systems and methods that use a raised floor constructiontypically do not efficiently meet the cooling requirements ofrack-mounted equipment. In particular, racks that include high-powerequipment having a thermal exhaust air output above 5,000 watts and upto 14,000 watts present a particular challenge for such systems andmethods. A raised floor construction typically provides an open floortile or a floor grill or vent having a venting area of about 12 by 12inches and is configured to deliver from about 200 cfm to about 500 cfmof cool air. A rack of high-power equipment drawing up to 10,000 wattsand requiring an air flow of approximately 1,800 cfm, therefore, wouldneed about 3.5 to about 5 open floor tiles, grills or vents disposedaround the rack's perimeter to supply sufficient cool air to meet itscooling requirements, with a 14 kW rack using up to about 2,240 CFM orabout 4.5 to about 11.2 floor tiles. Such a floor configuration would bedifficult to achieve in equipment moms crowded with racks, andimpossible to implement if racks are arranged side-by-side in rows. Aircooling systems and methods that incorporate raised floorconfigurations, thus, are typically only used with racks spaced apart toprovide sufficient floor area to accommodate multiple open floor tiles,grills or vents. For typical rack spacing, this places a limit on thedensity of equipment that can be achieved. When a raised floor is notused, distributing cool air from one or more centralized airconditioning systems is even more difficult, as the cool air typicallyis distributed across a room containing rows of racks.

Equipment rooms and data centers are often reconfigured to meet newand/or different equipment needs that require individual racks to berelocated and/or replaced. In this context, raised floor air coolingsystems and methods are inflexible and can typically only bereconfigured and/or retrofitted to service rearranged, relocated and/ornewly installed equipment racks at considerable cost. Raised floorconfigurations cannot easily and inexpensively accommodate the manner bywhich users typically deploy equipment racks and reconfigure equipmentrooms and data centers to meet their new or changing needs.

In addition, cooling systems and methods that use raised floorconstruction lack physical flexibility and portability to operativelyaccount for a wide variation in electrical power consumption betweendifferent racks in an equipment room, and, in particular, between racksand enclosures located in the same row. Cooling systems and methods thatrely upon raised floor air passageways and open floor tiles, grills orvents to supply cool air may not be able to easily and inexpensivelyvary or concentrate cool air to those high power racks that consumerelatively large amounts of electrical power and have a high thermal airexhaust output. In addition, newly installed equipment may draw moreelectrical power than replaced or existing equipment that may lead tothermal problem areas in functioning equipment rooms.

Further, with existing air conditioning solutions, hot spots can developin a room due to a lack of proper recirculation of exhaust air fromracks to the return side of a room air conditioner. This can cause racksto undesirably draw warm air into the racks. To attempt to overcome aircirculation problems, many room air conditioners are designed to providevery cool air of approximately 58° F. and receive return air having atypical temperature of approximately 78° F. With an output airtemperature of 58° F., it is often necessary to add a humidificationsystem to increase moisture in the air in a data center due to the highlevel of dehumidification created as a byproduct of over cooling theair. Such humidification systems can be expensive to install andoperate.

SUMMARY

In general, in an aspect, the invention provides a system for coolinggas heated by passing the gas over heat-producing equipment to cool theequipment, the system comprising a heat exchanger including a first heattransfer mechanism configured to transfer heat from the heated gas to afirst coolant, a first condensing module connected for fluidcommunication with the heat exchanger and including second and thirdheat transfer mechanisms, the first condensing module being configuredto transfer heat through the second and third heat transfer mechanismsfrom the first coolant to second and third coolants in the second andthird heat transfer mechanisms, respectively, the first condensingmodule comprising: a first cooling subsystem including the first heattransfer mechanism and being configured to receive heated vapor phasefirst coolant and to transfer heat from the first coolant to the secondcoolant in the first heat transfer mechanism to cool and condense theheated first coolant; and a second cooling subsystem including thesecond heat transfer mechanism and being configured to receive heatedvapor phase first coolant and to transfer heat from the first coolant tothe third coolant in the second heat transfer mechanism to cool andcondense the heated first coolant, the system further comprising atleast one processor electrically coupled to the first condensing moduleand configured to control which of the first and second coolingsubsystems cools and condenses the first coolant, and a distributionarrangement connected for fluid communication with the first condensingmodule and the heat exchanger and configured to transfer the firstcoolant from the first condensing module to the heat exchanger.

Implementations of the invention may include one or more of thefollowing features. The at least one processor is configured to actuatethe second cooling subsystem for cooling the heated first coolant if thefirst cooling subsystem is inoperable, and the system further comprisesan uninterruptible power supply, including a battery, coupled to thesecond cooling subsystem to provide battery power to the second coolingsubsystem. The system further comprises a backup condensing module thatis connected for fluid communication with the second cooling subsystemand that comprises an ice storage tank. The backup condensing module isconfigured to receive third coolant from the second cooling subsystemand to transfer heat between the third coolant and water in the icestorage tank to cool the third coolant if the water is cooler than thereceived third coolant, and to cool the water if the received thirdcoolant is cooler than the water. The at least one processor isconfigured to control operation of the first and second coolingsubsystems such that: if the first cooling subsystem is operational andit is currently desired to form ice in the backup cooling subsystem,then the second cooling subsystem is actuated to subcool the thirdcoolant below 32° F. and to provide the subcooled third coolant to thebackup condensing module; and if the first cooling subsystem isnon-operational and the second cooling subsystem is operational, thenthe second cooling subsystem is actuated to cool the third coolant usingthe ice in the backup cooling subsystem and to cool the first coolantusing the third coolant that is cooled by the ice. The at least oneprocessor is configured to cause the second cooling subsystem to beactuated to subcool the third coolant only if the first coolingsubsystem, in combination with other condensing modules if any, is ableto supply cooling of the first coolant in excess of a demand for coolingof the first coolant. The at least one processor is configured to causethe second cooling subsystem to be actuated to subcool the third coolantonly if the first cooling subsystem, in combination with othercondensing modules if any, is able to cool the first coolant in excessof a maximum anticipated demand for cooling of the first coolant. Thesystem comprises at least one condensing module in addition to the firstcondensing module, and wherein the at least one processor is configuredto cause the second cooling subsystem of the first condensing module tobe actuated to subcool the third coolant only if the at least onecondensing in addition to the first condensing module is able to supplycooling of the first coolant in excess of a maximum anticipated demandfor cooling of the first coolant. The backup cooling subsystem comprisesmultiple ice storage tanks and wherein the at least one processor isconfigured to determine that it is undesirable to form ice in the backupcooling subsystem if all of the ice storage tanks are considered full ofice.

Implementations of the invention may also include one or more of thefollowing features. The at least one processor is configured to controlthe first and second cooling subsystems and the distribution arrangementsuch that the first coolant is provided to the heat exchangers at adesired temperature and pressure. The at least one processor isconfigured to control the first and second cooling subsystem and thedistribution arrangement such that the third coolant is provided to theheat exchangers at a desired, constant temperature with differentamounts of the third coolant in the system.

In general, in another aspect, the invention provides a system forcooling gas heated by passing the gas over heat-producing equipment tocool the equipment, the system comprising a heat exchanger sectioncomprising a heat exchanger configured to receive a first coolant and totransfer heat from the heated gas to the first coolant, a coolingsubsystem configured to receive heated first coolant from the heatexchanger and to cool the heated first coolant, a distributionarrangement connected to the heat exchanger and the cooling subsystemand configured to transfer the cooled first coolant from the coolingsubsystem to the heat exchanger and to transfer the heated first coolantfrom the heat exchanger to the cooling subsystem, and at least oneprocessor coupled to the cooling subsystem, the heat exchanger section,and the distribution arrangement and configured to: determine a dewpoint of the gas associated with the heat exchanger; monitor a physicalcharacteristic of the first coolant relevant to saturation of the firstcoolant; and control supply of the first coolant to the heat exchangersuch that a combination of temperature and pressure of the first coolantentering the heat exchanger put the first coolant at a saturation pointof the first coolant with a first coolant temperatures being above thedetermined dew point temperature.

Implementations of the invention may include one or more of thefollowing features. The cooling subsystem further comprises a firstcoolant temperature sensor and a first coolant pressure sensorconfigured to monitor temperature and pressure of the first coolantexiting the cooling subsystem, the at least one processor being coupledto the first coolant temperature sensor and the first coolant pressuresensor and configured to regulate the cooling subsystem such that thetemperature and pressure of the first coolant exiting the coolingsubsystem are at desired levels. The heat exchanger section includesmultiple heat exchangers and the at least one processor is configuredto: determine respective dew points of the gas in respective vicinitiesof each of the heat exchangers; monitor the physical characteristic ofthe first coolant near an entrance to each of the heat exchangers; andcontrol supply of the first coolant to the heat exchangers such thatcombinations of temperature and pressure of the first coolant enteringrespective ones of the heat exchangers put the first coolant atsaturation points of the first coolant with respective first coolanttemperatures being above respective determined dew point temperatures.The system further comprises a heat exchanger temperature (HET) sensorand a heat exchanger humidity (HEH) sensor configured to monitor atemperature and a humidity of the heated gas disposed adjacent to theheat exchanger, the at least one processor being coupled to the HETsensor and to the HEH sensor and being configured to use temperature andhumidity indicia from the HET and HEH sensors to determine the at leastone dew point.

In general, in another aspect, the invention provides an ice storagesystem comprising an equipment rack housing of industry-standarddimensions for housing rack-mountable information technology equipment,an ice storage tank comprising: a tank housing configured to provide areservoir for holding water; a heat exchanger disposed in the tankhousing and configured to convey coolant and to transfer heat betweenthe water and the coolant; and a level indicator configured to monitorand provide a first indication of a level of water in the reservoir, thesystem further comprising an input line, disposed through the equipmentrack housing, configured to receive the coolant and being coupled to aninput of the heat exchanger, and an output line, disposed through theequipment rack housing, configured to convey the coolant from theequipment rack and being coupled to an output of the heat exchanger,where the ice storage tank is disposed inside the equipment rackhousing.

Implementations of the invention may include one or more of thefollowing features. The system comprises multiple ice storage tanks andeach ice storage tank includes at least one shutoff valve configured toselectively inhibit the coolant from flowing through the heat exchangerin response to a second indication that an amount of ice in thecorresponding reservoir is above a threshold amount. The systemcomprises multiple ice storage tanks and three ice storage tanks aredisposed in the equipment rack housing.

In general, in another aspect, the invention provides a system forcooling gas heated by passing the gas over heat-producing equipment tocool the equipment, the system comprising a heat exchanger configured toreceive a first coolant and to transfer heat from the heated gas to thefirst coolant, a cooling apparatus that includes a heat transfermechanism and that provides a chamber, the heat transfer mechanism beingconfigured to transfer heat from the first coolant disposed in thechamber to a second coolant disposed in the heat transfer mechanism, afirst shutoff valve connected to an output of the cooling apparatus forreceiving the first coolant and connected to an input of the heatexchanger to selectively permit flow of the first coolant from thecooling apparatus to the heat exchanger, a second shutoff valveconnected to an output of the heat exchanger and connected to an inputof the cooling apparatus to selectively permit flow of the first coolantfrom the heat exchanger toward the cooling apparatus, a third shutoffvalve coupled to the heat exchanger, and a pump arrangement connected tothe third shutoff valve and configured to draw gas and the first coolantthrough the third shutoff valve from the heat exchanger.

Implementations of the invention may include one or more of thefollowing features. The pump arrangement is configured to vent the drawngas to a region external to the system. The pump arrangement is furtherconnected to the cooling apparatus and is configured to convey the drawnfirst coolant to the cooling apparatus. The system comprises first,second, and third shutoff valves connected to the output of the coolingapparatus, the input of the cooling apparatus, and the pump arrangement,respectively, the system further comprising a processor coupled to thefirst, second, and third shutoff valves and to the pump arrangement, theprocessor being configured to control the shutoff valves and the pumparrangement such that: in response to an indication to add a new heatexchanger to the system with the new heat exchanger being coupled to aset of the first, second, and third shutoff valves, the processor willcause the first and second shutoff valves of the set to be closed, thethird shutoff valve of the set to be opened, the pump arrangement todraw gas from the new heat exchanger until a desired pressure isattained in the new heat exchanger and then the processor will cause thethird shutoff valve of the set to be closed, and the first and secondshutoff valves of the set to be opened; and in response to an indicationto remove a certain heat exchanger from the system with the certain heatexchanger being coupled to the set of the first, second, and thirdshutoff valves, the processor will cause the first and second shutoffvalves of the set to be closed, the third shutoff valve of the set to beopened, the pump arrangement to draw the first coolant from the certainheat exchanger until a desired pressure is attained in the certain heatexchanger and then the processor will cause the third shutoff valve ofthe set to be closed, and the first and second shutoff valves of the setto be opened.

In general, in another aspect, the invention provides a system forcooling gas heated by passing the gas over heat-producing equipment tocool the equipment, the system comprising a condenser including aplurality of heat transfer elements, a heat exchanger configured toreceive a first coolant from the condenser, to transfer heat from theheated gas to the first coolant, and to supply the heated first coolantto the condenser, a pump arrangement coupled to the condenser and theheat exchanger and configured to pump the first coolant from thecondenser to the heat exchangers, and cooling means for transferringheat from the first coolant through at least one of a plurality of heattransfer elements into at least one of a second coolant and a thirdcoolant.

Implementations of the invention may include one or more of thefollowing features. The cooling means is configured to select betweenwhich one of the second coolant and the third coolant to use to cool thefirst coolant. The cooling means includes primary cooling means forcooling the second coolant, and backup means for cooling the thirdcoolant. The system backup means comprises an ice storage tank, theprimary means is configured to cool the third coolant, and the coolingmeans is configured to direct the third coolant cooled by the primarymeans to the backup means to freeze water stored in the ice storagetank. The cooling means is configured to regulate amounts of the secondcoolant provided to the condenser to control at least a temperature ofthe first coolant pumped from the condenser. The system furthercomprises pump regulator means for regulating the pump arrangement tocontrol pressure of the first coolant such that the first coolantentering the heat exchanger will be at saturation while a temperature ofthe first coolant entering the heat exchanger is above a dew pointtemperature of the heated gas in a vicinity of the heat exchanger.

In general, in another aspect, the invention provides a method ofcooling information technology equipment using a heat exchanger, themethod comprising sensing a humidity of gas in a vicinity of the heatexchanger, sensing a gas temperature of the gas in the vicinity of theheat exchanger, determining, from the humidity and the gas temperature,a dew point temperature of the gas in the vicinity of the heatexchanger, determining a saturation point combination of saturationcoolant temperature and saturation coolant pressure for a coolant to besupplied to the heat exchanger such that the saturation coolanttemperature is above the dew point temperature and the coolant issaturated, and supplying the coolant to the heat exchanger such that thecoolant has the determined combination of saturation coolant temperatureand saturation coolant pressure entering the heat exchanger.

Implementations of the invention may include one or more of thefollowing features. Supplying the coolant includes adjusting cooling ofthe coolant such that the temperature of the coolant entering the heatexchanger is at about the saturation coolant temperature.

In general, in another aspect, the invention provides a system forcooling gas heated by heat-producing electronic equipment, the systemcomprising a heat exchanger configured to transfer heat from the heatedgas to a first coolant, a first cooling module connected for fluidcommunication with the heat exchanger and including a first condenserconfigured to cool and condense incoming first coolant from vapor toliquid, the first cooling module being configured to transfer heat fromthe first coolant to a second coolant to cool the first coolant. asecond cooling module connected for fluid communication with the heatexchangers and including a second condenser configured to cool andcondense incoming first coolant from vapor to liquid, the first coolingmodule being configured to transfer heat from the first coolant to athird coolant to cool the first coolant, and a condenser-chargecontroller-configured to regulate a first coolant liquid level in thefirst condenser.

Implementations of the invention may include one or more of thefollowing features. The condenser-charge controller comprises first andsecond condenser-charge controller subsystems connected and configuredto control liquid levels in the first and second condensers,respectively. The first and second subsystems each include a liquidlevel sensor configured to determine a liquid level in the respectivecondenser, a pump, and a controller coupled to the pump and the liquidlevel sensor and configured to regulate the pump to affect thecorresponding liquid level. The liquid level sensor is a pressuredifferential sensor and the cooling module includes a coolant containerconnected to the condenser and the pump and the liquid level sensor isconnected to the coolant container to determine the liquid level in thecoolant container, the liquid level in the coolant container beingrelated to the liquid level of the condenser. The first and secondcooling modules further comprise a container connected to the condenserand configured to store the first coolant, a pump connected to thecontainer and configured to pump the first coolant from the container, apurge mechanism connected to the pump, a purge controller coupled to thepurge mechanism and configured to actuate the purge mechanism to purgeat least some of the first coolant pumped by the pump, a fill mechanismconnected to the container, and a fill controller coupled to the fillmechanism and configured to actuate the fill mechanism to supply liquidfirst coolant to the container. The purge controller is configured toactuate the purge mechanism if the pump is operating at about fullcapacity and the liquid level of the container rises above an upperthreshold level and/or more than a first threshold amount. The fillcontroller is configured to actuate the fill mechanism if the pump isoperating at about minimum capacity and the liquid level of thecontainer drops below a lower threshold level and/or more than a secondthreshold amount.

In general, in another aspect, the invention provides a data centercooling system comprising equipment racks configured to house datacenter equipment, the racks being arranged in rows such that equipmentdisposed in the racks will vent hot air into an aisle defined betweenthe rows of racks, and a heat exchanger unit including s heat exchangerconfigured to draw in and cool air from the aisle, the heat exchangerunit including: a housing configured to contain the heat exchanger, anda mounting apparatus connected to the housing and to at least one rack,the mounting apparatus configured such that the heat exchanger isdisposed at least partially vertically aligned with the aisle.

Implementations of the invention may include one or more of thefollowing features. The mounting apparatus is configured such that theheat exchanger is disposed at least partially directly over the aisle.The mounting apparatus is configured such that the heat exchanger isdisposed substantially entirely directly over the aisle. The mountingapparatus is configured to connect to at least one rack in each of twodifferent rows of the equipment racks such that the heat exchanger unitstraddles the aisle.

In general, in another aspect, the invention provides a system forcooling gas heated by passing the gas over heat-producing equipment tocool the equipment, the system comprising a heat exchanger including aheat exchanger heat transfer mechanism configured to transfer heat fromthe heated gas to a heat exchanger coolant, a first condensing moduleconnected for fluid communication with the heat exchanger and includinga first heat transfer mechanism, the first condensing module beingconfigured to transfer heat through the first heat transfer mechanismfrom the heat exchanger coolant to a first coolant in the first heattransfer mechanism, and a second condensing module connected for fluidcommunication with the heat exchanger and including a second heattransfer mechanism, the second condensing module being configured totransfer heat through the second heat transfer mechanism from the heatexchanger coolant to a second coolant in the second heat transfermechanism.

Implementations of the invention may include one or more of thefollowing features. The first and second condensing modules are coupledin parallel through a single coolant loop to the heat exchanger. Thesystem further comprises a processor coupled to the first and secondcondensing modules and configured to assign a cooling task to each ofthe condensing modules based upon expected cooling demand for the heatexchanger coolant and cooling capacities providable by at least thefirst and second condensing modules. The processor is configured toassign the first condensing module as a primary module for cooling theheat exchanger coolant, to assign the second condensing module as a lagmodule, for cooling the heat exchanger coolant, if the expected coolingdemand exceeds a cooling capacity providable by the first condensingmodule, and to assign the second condensing module as a redundant moduleif the cooling capacity of the first condensing module plus the coolingcapacity of any lag condensing modules is at least as great as theexpected cooling demand, the redundant module being designated for usein cooling a backup refrigerant if the primary module and any lagmodules are operational and for cooling the heat exchanger coolant ifthe primary module or any lag modules are not operational. The primarycondensing module is used to cool the heat exchanger coolant unless theprimary condensing module is inoperative, the lag module, if any, isused to cool the heat exchanger coolant if the cooling demand exceedsthe cooling capacity of the primary module and any other lag module, andthe redundant module, if any, is used to cool the heat exchanger coolantif any of the primary and lag, if any, modules is inoperative and thecooling demand exceeds the cooling capacity of the operative primary andlag, if any, modules, and is used to produce ice if the cooling capacityof the operative primary and lag, if any, modules at least meets thecooling demand.

Implementations of the invention may also include one or more of thefollowing features. The system is configured to use excess coolingcapacity of the second condensing module to produce ice for use incooling the heat exchanger coolant if the first condensing module isinoperative. The second heat transfer mechanism is further configured totransfer heat through the second heat transfer mechanism from the heatexchanger coolant to a third coolant in the second heat transfermechanism. The system further comprises an ice storage tank connected tothe second heat transfer mechanism, where the second condensing moduleincludes a battery and a pump connected to the battery and the icestorage tank, and where the battery can be used to power the pump tocirculate the third coolant between the second heat transfer mechanismand the ice storage tank to cool the third coolant with the ice and tocool the heat exchanger coolant with the third coolant. The first andsecond condensing modules each have a cooling capacity that is nogreater than an expected cooling demand for the heat exchanger coolant.The first and second condensing modules each have a cooling capacitythat is at least as great as an expected cooling demand for the heatexchanger coolant.

Various aspects of the invention may provide one or more of thefollowing advantages. A highly-available, fault-tolerant IT coolingsystem may be provided. Heat exchanger modules can be easily addedand/or removed from a cooling system. Varying cooling needs can beaccommodated. A multi-vector cooling system may be provided. Three, andpossibly more, subsystems can be provided and selectively used to coolcoolant for cooling hot equipment exhaust gases. Equipment can be cooledduring power failures using battery power and thermal ice storage.Applications of high-densities of heat-producing equipment can besupported. Hot equipment exhaust gas can be cooled efficiently. Hotequipment exhaust gas can be cooled without substantially mixing the hotexhaust gas with cooler ambient gas. Heat-producing equipment can becooled without introducing moisture to cooled exhaust gas. Load changesof heat-producing equipment, e.g., in heat produced by a particularpiece of equipment, a change in number of pieces of equipment, and/or achange in location of equipment, can be accommodated. High-densitycooling can be provided, e.g., for high-density equipment installations.Cooling capacity demand increases (e.g., incremental increases) can beaccommodated. Cooling capacity can be modulated to better match thermalrequirements. N+1 redundancy can be provided.

These and other advantages of the invention, along with the inventionitself, will be more fully understood after a review of the followingfigures, detailed description, and claims.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a simplified block diagram of a cooling system according tothe invention.

FIG. 2 is a simplified diagram of a cooling module of the system shownin FIG. 1.

FIG. 3 is a simplified diagram of a coolant distribution section and aportion of a cooling module of the system shown in FIG. 1.

FIG. 4 is a simplified diagram of a heat exchanger module section of thesystem shown in FIG. 1.

FIG. 5 is a simplified diagram of a backup coolant cooling section ofthe system shown in FIG. 1.

FIG. 6 is a block flow diagram of a process of selecting a coolingmechanism for cooling coolant for use in cooling IT equipment using thesystem shown in FIG. 1.

FIG. 7 is a block flow diagram of a process of cooling IT equipmentusing the cooling module as shown in FIG. 2, the coolant distributionsection shown in FIG. 3, and the heat exchanger module section shown inFIG. 4.

FIG. 8 is a block flow diagram of a process of cooling primary coolantusing portions of the cooling module shown in FIG. 2.

FIG. 9 is a block flow diagram of a process of making ice for backupcooling of coolant using the cooling module shown in FIG. 2 and thebackup cooling section shown in FIG. 5.

FIG. 10 is a block flow diagram of a process of cooling secondarycoolant using the backup coolant cooling section shown in FIG. 5.

FIG. 11 is a block flow diagram of a process of adding a heat exchangermodule to the heat exchanger module section shown in FIG. 4.

FIG. 12 is a simplified diagram of portions of the system shown in FIG.1 for adding/removing heat exchanger modules.

FIG. 13 is a block flow diagram of a process of removing a heatexchanger module from the heat exchanger module section shown in FIG. 4.

FIG. 14 is simplified top view of an exemplary mechanical layout of thesystem shown in FIG. 1.

FIG. 15 is a block flow diagram of a process of assigning duties tocooling modules of the system shown in FIG. 1.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Embodiments of the invention provide techniques for cooling IT(Information Technology) equipment in the data center environment.Exemplary embodiments of the invention include one or more maincondensing modules, a coolant distribution section, a heat exchangermodule section, and a backup coolant section. The coolant distributionsection includes a bulk storage tank, an evacuation/recovery pump, amanifold and hoses. The condensing modules section sends cool liquid tothe heat exchanger module section by means of the distribution section,where the liquid is evaporated, vapor phase, into gas by hot air fromthe IT equipment, and the vapor coolant is returned to the maincondensing module(s). At the main condensing module(s), a primarycooling portion cools the heated vapor coolant back into a liquid forsupply to the heat exchanger module section by the distribution section.In the case of a failure to one of the primary condensing modules, asecondary condensing module can cool and condense the heated vaporcoolant if power has not failed to the system. If power has failed tothe system, the backup coolant section that includes several ice storagetanks can continue to cool, without using high power consumption vaporcompression systems, the heated coolant from the heat exchange modulesection for the duration of battery life or depletion of ice storage ofthe system. Other embodiments are within the scope of the invention.

Referring to FIG. 1, an IT cooling system 10 includes main condensingmodules 12, a processor 13, a backup section 14, a coolant distributionand storage section 16, and a heat exchanger (HE) module section 18.While three condensing modules 12 are shown in FIG. 1, other quantitiesof the modules 12 (e.g., one or two) may be used. The system 10 isconfigured to cool hot IT exhaust air in the HE module section 18 bysupplying HE coolant from the coolant distributor 16 that is cooled andcondensed by the condensing modules 12. The modules 12 cool and condensethe vapor coolant using a primary condenser/evaporator (C/E) 20 inconjunction with a vapor compressor 24 and a condenser 26. The system10, however, may cool and condense the HE coolant using the backupsection 14. The backup section 14 here is an ice storage and glycoldistribution section, but it is possible that the backup section 14could be a chilled water system acting as either the primary or backupsystem. The modules 12 may be connected to the backup section 14 througha single pair of input output lines or multiple lines (e.g., a paircorresponding to each module 12). The secondary condenser 22 may be usedin cases where there is a failure to the primary condenser 20 but not apower failure, while the backup section 14 may be used if there is apower failure affecting the primary condenser 20. The secondarycondenser 22 is configured to operate as an evaporator interacting withthe primary condenser 20 in a standby-mode to form ice in the icestorage section 14 for future use in cases of one or more failures(e.g., power loss) affecting the primary condenser 20.

The processor 13 is connected to the modules 12 and the sections 14, 16,18 for monitoring and controlling operation of the system 10. Functionsdescribed below relating devices (e.g., PID controllers) and/or tocontrol of devices noted may be performed by measuring appropriatecharacteristics of the system, monitoring measurements by the processor13, having the processor 13 manipulate the measurements, and having theprocessor 13 provide appropriate control signals to appropriate devices.The processors 13 may be physically disposed within one of the modules12 or the sections 14, 16, 18. Further, more than one processor may beused, and the processors disposed at different locations, e.g., inmultiple ones, and possibly all, of the modules 12 and/or sections 14,16, 18. The discussion below refers only to the processor 13, but otherembodiments are included in the discussion by reference to the processor13.

Referring to FIG. 2, the main condensing module 12 includes the primaryC/E 20, the secondary/ice generation C/E 22, the vapor compressor 24,and the condenser 26. The cooling module 12 is configured to coolcoolant that is heated in the HE module section 18 (FIG. 1), condensethis coolant and provide the cooled, condensed coolant for return to theHE module section 18 for use in cooling further hot air, e.g., producedby IT equipment. The coolant received from the HE module section 18 andcooled by the module 12 is referred to as HE coolant, and may be any ofvarious coolants, preferably R134A. Coolant used between the primary C/E20, the compressor 24, and the condenser 26 is referred to as primarycoolant and may be any of a variety of coolants, preferably R410A. Also,fluid used between the secondary C/E 22 and the backup section 14 isreferred to as secondary coolant and may be any of a variety ofcoolants, preferably a glycol-water solution. The module 12 is furtherconfigured to chill the glycol solution, e.g., during times when theparticular module 12 is not used to condense heated IT air, and providethis chilled solution to the backup section 14, that includes icestorage tanks to produce/store ice for backup cooling of the HE coolant.The module 12 is further configured to melt the ice, e.g., duringfailures of the primary C/E 20, in the backup section 14 to cool andcondense the HE coolant for cooling the heated IT air. The module 12provides redundancy through the redundant C/Es 20, 22 as desired (e.g.,if the C/E 20 is unavailable, the C/E 22 can still provide condensing).

The primary C/E 20, the vapor compressor 24, and the condenser 26 areconnected and configured to cool/condense the vapor HE coolant receivedin a line 30 as regulated by an isolation valve 31. The primary C/E 20is configured to absorb heat from the hot vapor HE coolant through aheat exchanger into the primary coolant (“hot” being used as a relativeterm as the vapor HE coolant is typically <70° F.). The vapor HE coolantenters the primary C/E 20 through an input 32, is cooled by the primaryC/E 20, and cool, liquid HE coolant exits through an output 34 into anHE coolant container 36. Heat is transferred from the HE coolant to theprimary coolant, thereby cooling the HE coolant and heating the primarycoolant. The heated primary coolant passes from the primary C/E 20 tothe compressor 24. The compressor 24 is configured to mechanicallyincrease the pressure of the primary coolant such that the primarycoolant entering the condenser 26 is at a saturation temperature abovethe temperature of fluid entering the condenser from the supply line 38.The compressor 24 is connected to the condenser 26 to pass thehigh-pressure (e.g., about 440 psi) high-temperature (e.g., about 190°F.) primary coolant to the condenser 26. The condenser 26 is configuredto cool and condense the primary coolant through heat exchange (via heatexchanger plates) with condenser water supplied via a line 38 andexpelled through a line 40, with the lines 38, 40 connected by anisolation valve 42. The amount of water supplied through the line 40 isregulated, to maintain design pressures, by a regulator valve 44connected to the intake line 38. The condenser 26 cools the primarycoolant sufficiently below saturation to induce a phase change to leavethe primary coolant as a subcooled liquid. The condenser 26 is connectedto the primary C/E 20 and configured to supply relatively high-pressure(e.g., about 440 psi), relatively moderate-temperature (e.g., about 105°F.) sub-cooled liquid primary coolant to the primary C/E 20.

The HE coolant container 36 is configured and disposed to retain HEcoolant at an equilibrium level between the primary and secondary C/Es20, 22. The container 36 is configured to hold liquid HE coolant flowingfrom the C/Es 20, 22. The container 36 is physically disposed with abottom 46 at a known elevation from the bottoms 48, 50 of the C/Es 20,22 and sized such that the container extends higher than a level of theHE coolant. The level of the HE coolant equalizes to the same level inthe C/Es 20, 22 and the container 36. This level is monitored by liquidlevel sensor 100 (FIG. 3). Additional HE coolant may be added to thecontainer 36 through a fill line 52 as controlled by a solenoid valve54. Furthermore excessive HE coolant may be purged back to bulk storagetanks by opening a solenoid valve 106 (FIG. 3). A circulation line 111is connected through the bottom 46 of the container 36 through which theliquid HE coolant can be circulated back to the HE module section 18.

The secondary C/E 22 is connected to an intake line 72 and an outputline 74 for receiving and providing the glycol solution from and to thebackup ice storage section 14 (FIG. 1). The C/E 22 may also be supportedby means of connection to an external chilled water system instead ofice storage, with the C/E 22 possibly functioning as the primary coolerof the vapor phase first coolant from the HEs 18. The lines 72, 74 areseparated by an isolation valve 76 and glycol coming into the secondaryC/E 22 is regulated by a regulation valve 78. Closure of the valve 76will facilitate 2-way operation of the regulation valve 78.

Various sensors and controllers are provided for monitoringcharacteristics and regulating operation of the module 12. A temperaturesensor 56 and a pressure sensor 58 are configured to monitorcharacteristics of the HE coolant in the container 36. Temperature andpressure values from these sensors 56, 58 are provided to an HE coolantpressure PID (proportional+integral+derivative) controller 60. The PIDcontroller 60 uses the received values to send control signals to theregulation valve 78 and to a glycol circulation pump 77. The signals tothe pump 77 cause the pump 77 to circulate glycol for ice production(described below). The PID controller 60 also sends control signals to avariable frequency drive 62 to regulate the HE coolant pressure, inconjunction with the HE coolant temperature, to help keep the HE coolantcondensing pressure sufficiently below a determined pressure associatedwith the HE coolant temperature. For example, the pressure is controlledsuch that the HE coolant temperature and pressure are at a desiredcombination when entering the HE modules 150 (e.g., at saturation withthe HE coolant temperature being above a dewpoint temperature of heatedIT exhaust air). Typically, the HE coolant pressure leaving the drive 62is about 5 psig below that of a desired heat exchanger evaporatorpressure, thus facilitating a pressure differential for transporting theprimary coolant from the heat exchanger section 18 back to thecondensing modules 12. The variable frequency drive 62 controls theoperating speed of the compressor 24 in response to the control signalsfrom the PID controller 60. The PID controller 60 also sends controlsignals to a regulation valve 78. A temperature sensor 64 and a pressuresensor 66 are configured to monitor characteristics of the heatexchanger coolant exiting from the primary C/E 20 on its way to thecompressor 24. The sensors 64, 66 provide indicia of the temperature andpressure of the heat exchanger coolant to a superheat PID controller 68.The PID controller 68 uses the indicia of temperature of suction gasfrom the sensor 64 to determine and send control signals to anelectronic expansion valve 70 to regulate the superheat of the suctiongas to maintain the superheat at a desired value (or within a desiredrange of values). Typically, the superheat is maintained between about5° F. and about 20° F. above saturation to help prevent possible liquidingestion by the compressor 24 and resulting damage to the compressor24. A pressure sensor 80 is configured and connected to measure thepressure of the high-pressure, high-temperature primary coolant on itsway to the condenser 26 from the compressor 24. Pressure values from thesensor 80 are used by a discharge pressure PID controller 82 todetermine and send control signals to the glycol flow control valve 44.These control signals cause the valve 44 to regulate glycol or waterflow to maintain the compressor discharge pressure at a desired level. Asight glass 71 and a filter dryer 73 are provided in the line connectingthe condenser 26 and the C/E 20 to allow viewing, and provide cleaningand dehydration, respectively, of the primary coolant.

Referring to FIG. 3, the HE Coolant distribution/storage section 16 isphysically intertwined with portions of the condensing modules 12 andincludes a receiver/reservoir 90, a temperature sensor 92, a pressuresensor 94, a reclaim/vacuum pump 126, and a circulation pump 98. Each ofthe condensing modules 12 has a circulation pump 98, with only one ofthe cooling modules 12 shown in FIG. 3. The distribution section 16 isconfigured to pump the heat exchanger coolant from the condenser 12(FIG. 2) to the heat exchanger module section 18 (FIG. 1). Thedistribution section 16 is further configured to evacuate heat exchangermodules that are newly connected to the heat exchanger module section18, and to reclaim heat exchanger coolant from a heat exchanger moduleto be removed from the HE module section 18.

The circulation pump 98, the pressure differential sensor 100, a liquidlevel PID controller 102, variable frequency controller 104, and theliquid purge valve 106, are configured to provide the HE coolant to theHE module section 18 via a liquid output line 110 at desired pressureand temperature. The pump 98 is connected to the outlet line 110 fromthe container 36. The line 111 feeding the line 110 includes aswing-type check valve 114 to help ensure one-directional flow of the HEcoolant in the feed line 111. The line 110 includes a sight glass 112for viewing the HE coolant and a filter/dryer 116 that is configured toclean and dehydrate the HE coolant. The sensor 100 is configured toprovide an indication of pressure differential between a top 45 and thebottom 46 of the container 36. This pressure difference is proportionalto the liquid level in the container 36, which is even with the liquidlevels in the C/Es 20, 22. The pressure difference indication isprovided to the PID controller 102 that produces and sends controlsignals to the variable frequency controller 104. The controller 104sends signals to the pump 98 to regulate the speed of the pump 98 tomaintain a desired liquid level in the container 36 and the C/Es 20, 22.This level is maintained despite varying mass flow rates in the primaryC/E 20, vapor compressor 24, and condenser 26 due to fluctuating heatloads managed by the heat exchanger section 18. The output HE coolantfrom the pump 98 feeds into the common output line 110 for conveying theHE coolant to the HE module section 18. If the liquid level remainshigh, or rises more than a threshold amount (e.g., rises at all), and/orabove a threshold level, after the pump 98 has been operating at or nearfull speed (capacity), then the purge valve 106 will be opened by theprocessor 13 to direct some of the HE coolant to the reservoir 90through a line 122. Alternatively, a mechanical pressure-activated valvemay be used that senses excessive discharge pressure at the pump 98 andopens to divert flow into the reservoir 90, replacing theelectrically-controlled solenoid valve 106.

HE coolant may be added to any of the primary coolant modules 12 asdesired by operation of the liquid fill solenoid 54. With the fillsolenoid 54 open, HE coolant from the reservoir 90 will be gravity-fedinto a fill line 124. HE coolant in the fill line 124 is provided to thecontainer 36 in each coolant module 12 whose fill solenoid 54 is open.The solenoid 128 is controlled by the processor 13 to supply coolant tothe container 36 if the pump 98 is operating at or near minimum capacityand the liquid level in the container 36 drops more than a thresholdamount (e.g., drops at all) and/or below a threshold level.

The level sensor 100, the pump 98, the purge valve 106, the solenoid 54,and the processor 13 serve as a liquid level control system. The liquidlevel in the container 36 is regulated by the purge valve 106 and thesolenoid 54 under the control of the processor 13 depending upon theoperation of the pump 98 and the status of the liquid level. This systemhelps to ensure that the liquid level stays within a desired range andthat hence each cooling module 12 handles a fair share of the coolingload. Thus, this system helps provide a condenser-charge optimizationsystem that helps optimize usage of the multiple cooling modules 12 anddistribute cooling load somewhat evenly.

The distribution/storage section 16 further includes a reclaim solenoid128 and a vacuum solenoid 130 in addition to the reclaim/vacuum pump126. The reclaim/vacuum pump 126 is configured to assist with removingand adding HE modules from/to the HE module section 18. The pump 126 isconnected to recovery/vacuum line 132 that is connected to the HE modulesection 18. The pump 126 is configured to pump liquid or vapor phase HEcoolant through the line 132 from the HE module section 18 and outputthe liquid coolant through the reclaim solenoid 128 to the reservoir 90(with the reclaim solenoid 128 open and the vacuum solenoid 130 closed).The pump 126 is further configured to pump gases to evacuate an interiorchamber of an HE module of the module section 18. The pump 126 can pumpgases through the line 132 from the HE module section 18 and exhaust thepumped gases to the atmosphere through the vacuum solenoid 130 (with thevacuum solenoid 130 open and the reclaim solenoid 128 closed).

Heated gas HE coolant to be cooled, condensed, and returned to the HEmodule section is received from the HE module section 18 through a vaporline 134. The heated HE coolant is provided to the C/Es 20, 22 of eachof the primary condensing modules 12 whose vapor isolation solenoid 31is open.

Referring to FIGS. 2-3, the module 12 can be set to provide chilledsecondary coolant to the backup ice storage section 14 (FIG. 1) toproduce ice for backup cooling of the HE coolant. For this mode ofoperation, the vapor isolation valve 31 is closed such that hot, vaporHE coolant is not provided to the C/Es 20, 22. In this mode, the C/Es20, 22 work in tandem to cool the secondary coolant, e.g., a glycolsolution, and provide the glycol solution to ice storage tanks. Theliquid level in the container 36 is adjusted as desired for performanceby opening the liquid fill solenoid 54 or opening the liquid purgesolenoid 106 and running the circulation pump 98. In ice generationmode, the C/Es 20, 22 and the glycol circulation pump 77 are activated.The C/E 20 operates as a condenser and the C/E 22 operates as anevaporator. The primary HE coolant is passed between the devices 20, 22,with the device 20 cooling the HE coolant to well below the 32° F.freezing point of water, e.g., 0° F. Heat of condensation in the device20 is transferred to the primary coolant system 20, 24, 26. The HEcoolant circulated to the device 22 will undergo a phase change in thedevice 22, absorbing heat from the glycol solution and evaporating. Theheat of evaporation absorbed from the glycol solution by means of heattransfer in the device 22 is the heat of fusion liberated from the icestorage tanks (see FIG. 5 and description below).

Referring to FIG. 4, the HE module section 18 includes a set of heatexchanger modules 150 connected to the liquid input line 110, therecovery/vacuum line 132, and the vapor line 134 via a liquid shutoffvalve 152, a recovery/evacuation isolation valve 154, and a vaporisolation valve 156. While three modules 150 are shown, other quantitiesof the modules 150 (e.g., one, two, four, five, etc.) may be used. Themodules 150 are preferably connected to the isolation valves 152, 154,156 with flexible hoses. The functionality of the valves 152, 154, 156,and 168 may be combined into a multi-function, multi-way valve blockmounted to the header assembly 110, 132, 134. The shutoff valves 152,154, 156 can be any of a variety of valves, e.g., electric solenoidvalves, with the valve 152 inhibiting flow of coolant prior to propermodule commissioning (FIG. 11); The heat exchanger module section 18 isconfigured to transfer heat from hot IT equipment exhaust air into theHE coolant, supplied via the liquid line 110, which becomes heated andis expelled through the vapor line 134 to the C/Es 20, 22 (FIGS, 1, 3).

The heat exchanger modules 150 each include a set of fans 158 (shownrepresentatively as a single fan), input temperature and humiditysensors 160, 162, an output air temperature sensor 164, a pressuresensor 166, a pressure regulator valve 168, finned heat exchanger tubes170, and a dew point calculator 172 and PID controllers 174, 176implemented by the processor 13 (FIG. 1). The dew point calculator 172implemented by the processor 13 is configured to monitor the temperatureand humidity of the input heated IT equipment exhaust air via thesensors 160, 162 to determine the dew point of incoming air. Based onthe temperature and humidity of the incoming air, the calculator 172further determines a pressure set point for incoming HE coolant andsupplies this set point to the PID controller 174. The PID controller174 uses the monitored HE coolant pressure indicated by the pressuresensor 166 as a process variable (PV) to determine a desired pressure ofthe HE coolant within the finned HE tube 170. The PID controller 174regulates the pressure regulator valve 168 to provide a desired HEcoolant evaporator pressure such that the pressure of the HE coolantwithin the heat exchanger tube 170 corresponds to a saturationtemperature that is slightly above the IT air dewpoint as determined bythe microprocessor 13. The HE coolant preferably does not inducecondensation of the air passing over the heat exchanger tube 170providing a sensible cooling process.

The processor 13 further monitors and regulates the output airtemperature from the heat exchanger modules 150. The PID controller 176monitors the temperature via the sensor 164 of the air exiting themodule 150. Based on this information, the PID controller 176 determinesan output that is provided to the PID controller 174 as a set pointoffset. The PID controller 174 uses the offset to control the flowregulator valve 168 to control the flow of the HE coolant through theheat exchanger tube 170 such that the output temperature of theexhausted air from the module 150 is at a desired temperature (or withina desired temperature range).

The modules 150 can be arranged in a variety of manners, e.g., toprovide total desired cooling capacity, to provide redundancy betweenthe modules 150, to provide both total desired cooling and redundancy,etc.

Referring to FIG. 5, the backup section 14 includes ice storage modules180 that each include an air pump 182, a differential pressure sensor184, a fluid level monitoring tube 186, an ice storage tank 188, afinned heat exchanger 190, and isolation valves 192, 194. Only one ofthe modules 180 is shown in detail in FIG. 5. Further, the number of icestorage modules 180 shown is exemplary and not limiting, with fewer ormore modules 180 being possible (e.g., further cascaded as shown). Theheat exchanger 190 is disposed in a reservoir of the tank 188 forholding water that may be converted to ice. The reservoirs 176 containwater while the secondary coolant comprises a glycol and water mixture.The tank 188 is connected to the line 74 (FIG. 2) through the motorizedball valve 192 to receive the secondary coolant for the heat exchanger190. The tank 188 is further connected to the line 72 (FIG. 2) throughthe ball valve 194 such that the secondary coolant can flow from theheat exchanger 190 to the line 72.

Ice quantity can be monitored using the air pump 182, the pressuresensor 184, and the monitoring tube 186. The tube 186 is submersed inwater in the tank 188 and pressurized using the air pump 182. The airpressure within the tube 186 displaces the water column in the tube 186with a small quantity of air escaping in the form of bubbles from anopening in a bottom of the tube 186. The air pressure within the tube isdirectly proportional to the column height of water displaced which isrelated to the amount of ice in the tank 188 (as more ice is formed, thewater expands and occupies more volume such that the liquid levelrises). The pressure sensor 184 monitors the pressure used to displacethe water from the tube 186. The monitored pressure is used to provideice quantity information (e.g., in real time) and is used by theprocessor 13 (FIG. 1) to regulate the motorized ball valve 192 toisolate tanks 188 from the glycol solution supply line 74 (e.g., forfully ice regenerated tanks 188). Thus, some ice tanks 188 may continueto receive the secondary coolant and continue to produce ice while othermodules 180 do not (e.g., helping to prevent over freezing in the tanks188 that fully regenerate before others).

Referring to FIG. 15, with further reference to FIG. 1, a process 410for assessing the cooling demands and capacity of the system 10 andallocating cooling modules 12 accordingly includes the stages shown. Theprocess 410 is exemplary only and not limiting. The process 410 may bemodified, e.g., by adding, removing, or rearranging the stages shown.The process 410 is preferably performed upon power up of the system 10,as indicated by stage 412. The process 410, however, may be performed atother times and in response to other events (e.g., a command from a userof the system 10).

The processor 13 uses programmed information about the module(s)12 andthe system 10 to make module assignments. The processor 13 is preferablyprogrammed with a maximum expected cooling demand and maximum coolingcapacity of the module(s) 12. Alternatively, the module(s) 12 mayprovide information to the processor 13 as to the cooling capacity ofthe corresponding module 12. Multiple cooling modules 12 preferablyprovide equal cooling capacity, although they may provide differentcapacities. The process 410 will yield assignments of the coolingmodule(s) 12 as primary, primary lag, or redundant. The primary coolingmodule 12 is the module 12 that is used to cool the HE coolant unlessthe module 12 is inoperative. A cooling module 12 is assigned as aprimary lag module if the maximum expected cooling capacity to besupplied exceeds the maximum cooling capacity of the primary module 12.The primary lag module 12 is preferably used only when the coolingdemand actually exceeds the cooling capacity of the primary module 12.Multiple modules 12 could be labeled as primary lag modules 12, witheach primary lag module 12 being assigned a priority number indicativeof the order in which each such module 12 is activated in response toincreasing cooling demand. A cooling module 12 is assigned as aredundant module if the maximum expected cooling capacity to be supplieddoes not exceed the maximum cooling capacity of the primary module(s)12. Multiple modules 12 could be labeled as redundant modules 12, witheach redundant module 12 being assigned a priority number indicative ofthe order in which each such module 12 is activated in response toincreasing cooling demand, or the order in which each redundant module12 is checked for operability. As an illustrative, but not limiting,example, the process 410 is shown with the system 10 having threecooling modules 12, each with 40 kW cooling capacity, and an expectedmaximum cooling demand of 80 kW.

At stage 414, the system 10 interrogates itself, with the processor 13polling the condensing modules 12. The processor 13 polls the modules 12to determine how many modules 12 are in the system 10. Depending onwhether the processor 13 determines that there are one, two, or three ofthe cooling modules 12, the process 410 proceeds to stage 416, 418, or420, respectively. The processor 13 is preferably programmed to know themaximum number of cooling modules 12 to expect, in this example three,so that if the determined count is zero or above the expected maximum,then the processor 13 re-polls the cooling modules 12.

At stage 416, the sole cooling module 12 is assigned to be the primarycooling modules 12. The sole cooling module 12 is labeled as the primarymodule 12 because it is the module 12 that is used to cool the HEcoolant unless the module 12 is inoperative.

At stage 418, with two cooling modules 12 in the system 10, an inquiryis made as to whether the maximum cooling demand of the system 10exceeds the cooling capacity, here 40 kW, of one of the cooling modules12. If the demand does not exceed the individual cooling capacity, thenat stage 422 the processor 13 assigns one of the modules 12 to be theprimary module and the other cooling module 12 to be a redundant coolingmodule. If the demand does exceed the individual cooling capacity, thenthe processor 13 assigns one of the modules 12 to be the primary moduleand the other module 12 to be a primary lag module.

At stage 420, with three cooling modules 12 in the system 10, an inquiryis made as to whether the maximum cooling demand of the system 10exceeds the cooling capacity, here 40 kW, of one of the cooling modules12. If the demand does not exceed the individual cooling capacity, thenat stage 426 the processor 13 assigns one of the modules 12 to be theprimary module, a second of the modules 12 to be a redundant #1 module,and the other cooling module 12 to be a redundant #2 cooling module. Ifthe demand does exceed the individual cooling capacity, then theprocessor 13 (knowing the maximum demand is 80 kW) assigns one of themodules 12 to be the primary module, a second of the modules 12 to be aprimary lag module, and the other module 12 to be a redundant module.

The flow of the process 410 is exemplary and not limiting. For example,if the system 10 contained more than three cooling modules 12, theninquiries could be made for different demand thresholds, such as whetherdemand exceeds capacity of one module 12 and whether demand exceedscapacity of two modules 12. As another example, to make moduleassignments, the processor 13 could assign a first module 12 as theprimary module, and then compare the maximum expected cooling demandwith the cooling capacity of the primary module 12. If the demandexceeds the capacity, then the processor 13 could assign other modules12 as primary lag modules 12 as available and as needed (with prioritiesfor multiple lag modules 12) until the maximum cooling capacity of theprimary modules 12 meets or exceeds the maximum expected cooling demand.Remaining cooling modules 12, if any, would be assigned as redundantmodules 12 with priorities assigned to multiple redundant modules 12.Still other techniques for allocating the cooling modules 12 could beused.

Referring to FIG. 6, with further reference to FIGS. 1-5, a process 200for operating the system 10 to allot cooling modules 12, to cool heatedIT exhaust air, and to make ice for backup cooling of the heated ITexhaust air includes the stages shown. The process 200 is exemplary onlyand not limiting. The process 200 may be modified, e.g., by adding,removing, or rearranging the stages shown.

At stage 202, an inquiry is made as to whether the present load coolingdemand exceeds the cooling capacity of the primary cooling module(s) 12.For each module 12, the processor 13 determines whether the primary C/E20 is operational, that is, receiving adequate power and is not broken.The processor 13 determines whether there has been a catastrophicfailure to the primary C/E 20 that would inhibit the C/E 20 fromproviding adequate cooling of the HE coolant for the HE module section18 (e.g., inadequate power supply, malfunction, etc.). The processor 13determines which primary cooling module(s) 12 are operational and thecooling capacity provided by the operational primary cooling module(s)12, if any. The processor 13 compares the current load demand againstthe capacity of the operational primary cooling module(s) 12. If thedemand exceeds the capacity (e.g., if a primary cooling module 12 isfailing/not operational), then the process 200 proceeds to stage 214. Ifthe demand may be met by the operational primary cooling module(s) 12,then the process 200 proceeds to stage 204.

At stage 204, the primary cooling module(s) 12, in particular the C/E(s)20 is(are) used to cool the HE coolant. In each primary module 12, thecompressor 24 and the condenser 26 supply cool primary coolant to theC/E 20 to cool the HE coolant flowing through the C/E 20. The cooled HEcoolant is then used by the HE module section 18, as described belowwith reference to FIG. 7, to cool the hot IT exhaust air.

At stage 206, with the primary module(s) cooling the IT air, theprocessor 13 inquires as to whether a redundant cooling module isavailable and operational. The processor 13 determines whether anyredundant modules exist in the system 10 as determined/assigned in theprocess 410, and if so whether the redundant module(s) 12 is(are)operational. The processor 13 can determine, e.g., whether there isadequate power, whether the pumps 98 are operational and whether thesecondary C/E 22 is operational.

At stage 208, the processor 13 inquires as to whether it is currentlydesired to prepare the backup coolant (here, ice) in the backup coolingsection 14. The processor 13 may, for example, determine whether it iscurrently an off-peak time for utility power use, whether the currentprice for power use is at a desirable rate, and/or whether the currenttime of day and/or day of week is desirable (e.g., a weeknight or aweekend). The processor 13 further preferably determines whether any ofthe ice tanks 188 are below a desired level of ice. The processor 13communicates with the level indicators 184 of the modules 180 todetermine the levels of ice in the tanks 188 and determines whether anyof these levels is below what is desired. If any level is below adesired level, then the processor 13 can decide to make ice. If theprocessor 13 determines that it is not desirable to prepare the backupcoolant, then the process 200 proceeds to stage 210. At stage 210, theprocessor 13 ensures that power is off to the secondary C/E 22 of theredundant module(s) 12. If, however, the processor 13 determines that itis desirable to prepare the backup coolant, then the process 200proceeds to stage 212.

At stage 212, the backup coolant is prepared, here ice being formed. Theprimary and secondary C/Es 20, 22 of the operational redundant module(s)12 are operated in conjunction with the backup ice storage coolingsection 14 to produce ice in the ice storage tanks 188. If the secondaryC/E(s) 22 is(are) non-operational for any reason (e.g., lack of power orhaving a catastrophic or other failure making operation undesirable),then the backup coolant will not be prepared (e.g., ice not formed foruse in cooling).

At stage 214, having determined that the primary module(s) cannot meetthe cooling demand, the processor 13 makes an inquiry as to whether asufficient combination of operational redundant and primary modules 12exists for meeting the cooling demand. The redundant modules 12 arechecked in succession of their priority numbers if multiple redundantmodules 12 exist until sufficient total cooling capacity is reached forthe demand. If such a combination is determined, the process 200proceeds to stage 216 where the determined combination of primary and/orredundant modules 12 is used to cool the HE coolant and thus the hot ITexhaust air (as described with respect to FIG. 7). If a combination ofprimary and redundant modules 12 is not determined by the processor 13that will meet the cooling demand, then the process 200 proceeds tostage 218.

At stage 218, the processor 13 makes an inquiry as to whether asufficient combination of redundant, primary, and backup modules 12exists for meeting the cooling demand. The processor 13 determineswhether the cooling demand can be met by the operational primary andredundant modules 12, if any, plus any backup cooling module 12, thatis, a module 12 whose primary C/E 20 is non-operational but whosesecondary C/E 22 is operational. The power for the secondary C/E ispreferably provided by battery backup, e.g., from an uninterruptiblepower supply (UPS) 23 such that a utility power failure causing theprimary C/E 20 to be inoperable does not cause the secondary C/E to beinoperable. The processor 13 also determines whether there is sufficientbackup coolant (here ice) for cooling the secondary coolant for coolingthe HE coolant and thus the hot IT exhaust air. If the conditions do notprovide for sufficient cooling to meet the demand adequately, then theprocess 200 proceeds to stage 220 where the system 10 is used to provideas much cooling capacity as possible to sustain IT functionality as longas possible. If the conditions do provide for sufficient cooling to meetthe demand adequately, then the process 200 proceeds to stage 222.

At stage 222, the determined combination of primary, redundant, and/orbackup modules 12 are used to cool the HE coolant. The backup coolantice storage section 14 is used to cool the secondary coolant for thebackup modules 12. The secondary coolant is pumped through the icestorage tanks 188 to cool the secondary coolant while melting the ice inthe tanks 188. The secondary coolant is pumped through the C/E 22 ofeach backup module 12 for use in cooling the HE coolant that is thenused to cool the hot IT exhaust air.

The process 200 loops back to stage 202 to periodically determine againwhether the cooling demand can be met by the primary cooling module(s)12. If the state of the system 10 changes from the primary coolingmodule(s) 12 not being able to meet the demand to being able to meet thedemand, then any redundant or backup modules 12 previously used are shutdown (e.g., use of UPS battery power terminated), and the primarymodule(s) 12 used to cool the HE coolant.

Referring to FIG. 7, with further reference to FIGS. 1-4, 6, a process230 for cooling the HE coolant and the hot IT exhaust air (see stages204, 216, and 222 shown in FIG. 6) includes the stages shown. Theprocess 230, however, is exemplary only and not limiting. The process230 may be altered, e.g., by having stages added, removed, orrearranged.

At stage 232, hot, vapor HE coolant is cooled and condensed in thecondensing module 12. The vapor HE coolant enters the module 12 throughthe line 30 with the isolation valve 31 open. As the HE coolant flowsinto the C/E 20 (stage 205) or the C/E 22 (stage 218), heat from thevapor is transferred through a heat exchanger in the respective C/E 20,22 that contains relatively cool primary or secondary coolant,respectively. Enough heat is transferred to cause the HE coolant tocondense to a sub-cooled liquid and to pool/collect at the bottom of therespective C/E 20, 22. The HE coolant is sub-cooled in that it is belowits saturation point, thus requiring heating before it wouldboil/evaporate/change phase from liquid to gas.

At stage 234, the liquid HE coolant is pumped by the coolant distributor16 to the HE module section 18. The sensor 100 provides an indication ofpressure differential between the top 45 and the bottom 46 of thecontainer 36. The pressure difference indication is used by the PI-Dcontroller 102- to produce and send control signals to the variablefrequency controller 104. The controller 104 sends signals to the pump98 to regulate the speed of the pump 98 to maintain a desired liquidlevel in the container 36 and the C/Es 20, 22. The circulation pump(s)98 of the operational cooling module(s)s 12 pumps the HE coolant throughthe liquid line 110 to the HE module section 18. The HE coolant may beviewed through the sight glass 112 in the line 110 to determine liquidquality and moisture content of the HE coolant. The liquid in the line110 is cleaned and dehydrated by the filter/dryer 116. The operatingpump(s) 98 increases the pressure of the HE coolant to a level that theHE coolant, after losing some pressure due to piping losses in transitto the section 18, pressure loss in the shutoff valve 156, and in thevalve 168, will be at a pressure such that the HE coolant will evaporateat a temperature that is above the dew point temperature of theexhausted IT air.

The processor 13 monitors the temperature and humidity of the IT exhaustair indicated by the sensors 160, 162 and determines the correspondingdew point of the exhaust air. Using the calculated dew point for each HEmodule 150, the processor 13 determines desired evaporation temperaturesfor the HE coolant, e.g., the dew point temperatures plus a safetymargin, e.g., of a few degrees Fahrenheit. The processor 13 uses thesetemperatures in conjunction with a known relationship between the HEcoolant's saturation point (evaporation point) and the coolant'stemperature and pressure to determine desired pressures of the HEcoolant within the HE tubes 170.

The processor 13 further monitors and regulates the output airtemperature from the heat exchanger modules 150 and determines a dewpoint offset. The PID controller 176 monitors the temperature via thesensor 164 of the air exiting the module 150. Based on this information,the PID controller 176 determines an output that is provided to the PIDcontroller 174 as a set point offset.

The PID controller 174 regulates the pressure of the HE coolant withinthe HE tube 170 to provide a desired output temperature of the exitingIT air while inhibiting condensation of the IT air. The PID controller174 uses the monitored HE coolant pressure indicated by the pressuresensor 166 as a process variable (PV), the set point provided by the PIDcontroller 172 related to the dew point of the IT air, and the set pointoffset provided by the PID controller 176 to determine a desiredpressure of the HE coolant within the filmed HE tube 170. The PIDcontroller 174 regulates the pressure regulator valve 168 to provide adesired pressure within the HE 150 such that the pressure andtemperature of the HE coolant in the heat exchanger tube 170. Thepressure as sensed by pressure sensor 166 is also regulated such thatthe temperature of the HE coolant is slightly above the dew point of thereturn air such that the HE coolant preferably will not inducesignificant condensation of moisture contained in the air passing overthe heat exchanger tube 170 providing a sensible cooling process. Thecontroller 174 also controls the flow regulator valve 168 to control theflow of the HE coolant through the heat exchanger tube 170 such that theoutput temperature of the exhausted air from the module 150 is at adesired temperature (or within a desired temperature range).

At stage 236, the HE coolant passes through the HE tubes 170, absorbsheat from the IT exhaust air passing by the tubes 170 and evaporates,thereby cooling the IT exhaust air. The HE coolant enters the module 150through the shutoff valve 152 and flexible tubing. The valve 168 isopened and closed as appropriate under the control of the PID controller174 to maintain the pressure and temperature of the HE coolant above thedew point of the IT air outside the tubes 170. The HE coolant changesphase from saturated liquid to saturated vapor, and increases inenthalpy, as it passes through the tubes 170. The latent heat from theIT exhaust air blown across the tubes 170 by the fans 158 is removed bypassing through the tubes 170 into the HE coolant, thus “cooling” the ITexhaust air (by removing the heat).

At stage 238, the HE coolant is returned to the cooling module 12. TheHE coolant is drawn from the tubes 170, through flexible piping, throughthe isolation valve 152, and through the vapor line 134. The gas-phaseHE coolant flows through the line 134 to the inlet of the cooling module12, and into the appropriate C/E 20, 22 for cooling/condensing. Theprocess 230 returns to stage 232.

Referring to FIG. 8, with further reference to FIGS. 1-2, and 6, aprocess 240 for implementing the stage 204 (FIG. 6) for providingsub-cooled liquid primary coolant to the C/E expansion valve 70 to coolthe HE coolant, and recycling primary coolant heated and evaporated inthe C/E 20 to be compressed, cooled and condensed includes the stagesshown. The process 240 is exemplary and not limiting. The process 240may be modified, e.g., by adding, removing, or rearranging stages.

At stage 242, high-pressure liquid primary coolant is supplied from theCondenser 26 to the primary C/E 20. The primary coolant passes from thecondenser 26 through the sight glass 71 for viewing of quality of theprimary coolant, and through the filter/dryer 73 for dehydration andcleaning. The pressure of the liquid primary coolant is dropped/reducedby the expansion valve 70 so that the primary coolant is saturated andready to change phase to a vapor. The primary coolant is metered intothe primary C/E 20 by the expansion valve 70.

At stage 244, the primary coolant changes phase into a vapor. Latentheat from the HE coolant passing through the C/E 20 transfers throughplates and is absorbed by the primary coolant. This heat causes theprimary coolant to convert from a saturated liquid to a super-heatedgas. The primary coolant increases in enthalpy while the HE coolantcorrespondingly decreases in enthalpy. The PID controller 68 monitorsthe temperature and pressure of the primary coolant exiting the C/E 20and actuates the expansion valve 70 such that the temperature andpressure of the primary coolant are at desired levels (superheated vaporleaving the C/E 20)

At stage 246, the primary coolant gas is provided to the compressor 24.The compressor 24 provides suction to draw the primary coolant from theC/E 20. Pressure values from the sensor 80 are used by the dischargepressure PID controller 82 to determine and send control signals to theglycol flow control valve 44. These control signals cause the valve 44to regulate glycol flow to maintain the compressor discharge pressure ata desired level such that the primary coolant entering the condenser 26is at the desired pressure. With this arrangement, a relativelyhigh-temperature (e.g., about 190° F.), relatively high-pressure (e.g.,about 445 psi) gas exits the compressor 24.

At stage 248, the primary coolant is cooled and condensed and suppliedto the primary C/E 20. The gas exiting the compressor 24 enters thecondenser 26. The condenser 26 cools the primary coolant by passing heatfrom the primary coolant to the coolant received from the line 38. Theprimary coolant leaves the condenser 38 as a relatively high-pressure(e.g., about 440 psi), relatively moderate-temperature (e.g., about 105°F.) sub-cooled liquid. This liquid is provided to the primary C/E 20.The process 240 returns to stage 242 to repeat the cycle ofheating/evaporating the primary coolant, cooling/condensing the primarycoolant, and recycling the primary coolant for heating, etc.

Referring to FIG. 9, with further reference to FIGS. 1, 2, 5, and 6, aprocess 260 for implementing the stage 210 (FIG. 6) for making ice inthe tanks 188 includes the stages shown. The process 260 for theice-generation mode of the sections 12, 14 is exemplary and notlimiting. The process 260 may be modified, e.g., by adding, removing, orrearranging stages.

At stage 262, the processor 13 actuates the pump(s) 77 to pump thesecondary coolant to the backup coolant modules 180. If a pump 77 is notoperational (at least desirably so), then the processor 13 reports afailure, deactivates the pump 77, and actuates a redundant pump. If thesecondary coolant is being adequately pumped, then the process 260proceeds.

At stage 264, the primary C/E 20 chills the HE coolant for use incooling the secondary coolant. The primary C/E 20 chills the HE coolantthat passes between the primary C/E 20 and the secondary C/E 22 with theisolation valve 31 shut to isolate the cooling module 12 from the HEmodule section 18. The HE coolant absorbs heat from the secondarycoolant through a heat exchanger in the secondary C/E 22 (acting as anevaporator) to cool the secondary coolant (e.g., to near 0° F.).

At stage 266, the chilled secondary coolant is routed to the ice storagemodules 180 to produce ice in the tanks 188. The secondary coolant flowsthrough the line 74 to the modules 180 and into the tanks 188. Whichtank(s) 188 receive secondary coolant and how much is controlled by theprocessor 13. The processor 13 monitors the levels of ice indicated bythe level indicator 184 of each of the tanks 188 and actuates thecontrol valves 192 associated with the tanks 188 to regulate the flow ofsecondary coolant through the tanks 188. As the coolant passes throughthe heat-transfer coils 190 in the tanks 188, heat is transferred fromthe water in the tanks 188 into the glycol-water secondary coolant suchthat the water freezes, forming ice. The formation of ice causes thelevel of the water in the corresponding tanks 188 to rise, increasingthe pressure to overcome in the tube 186 by the air pump 182. When thelevel rises to a desired level, as detected by the level detector 184 ofthe tanks 188, then the flow of secondary coolant to that tank 188 isstopped. When the levels of all the tanks 188 reach desired levels, thenthe ice-making mode is preferably terminated. The ice making may beterminated at any time if the processor 13 determines that it is nolonger desirable to snake ice (e.g., it is now peak time for powercost).

At stage 268, the secondary coolant is recycled. The secondary coolantleaving the tanks 188 flows back through the valve 194, the line 72, andback to the secondary C/E 22 and the tanks 188.

Referring to FIG. 10, with further reference to FIGS. 1, 2, 5 and 6, aprocess 270 for implementing the stage 216 (FIG. 6) for chilling thesecondary coolant and the HE coolant using the ice in the tanks 188includes the stages shown. The process 270 is exemplary and notlimiting. The process 270 may be modified, e.g., by adding, removing, orrearranging stages.

At stage 272, the processor 13 actuates the pump(s) 77 (for exemplarypurposes, multiple sections 12 are assumed) to pump the secondarycoolant. If the pumps 77 cannot adequately pump the secondary coolant toprovide sufficient flow of the secondary coolant, then the processor 13reports a failure and selects the alternate redudant pump 77. If thesecondary coolant is being adequately pumped, then the process 270proceeds. The actuated pump(s) 77 pumps the secondary coolant to the icetanks 188.

At stage 274, the secondary coolant is routed to the ice tanks 188 (forexemplary purposes, multiple tanks 188 are assumed). The motorized ballvalves 192 are controlled by the processor 13 to route the hot secondarycoolant to desired tanks 188, e.g., the tanks 188 that have sufficientice to cool the secondary coolant. The secondary coolant (which isheated in the C/E 22 as described below) is routed through the isolationvalves 192 under the control of the processor 13 to the ice tanks 188.The hot secondary coolant is passed through the tanks 188 where heattransfers from the coolant through the heat exchanger coils 190 into theice. The transferred heat melts the ice while the coolant is cooled bythe loss of heat.

At stage 276, the cooled coolant is routed through the secondary C/E 22to cool the HE coolant. The secondary coolant leaves the tanks 188 andis pumped by the pumps 77 to the secondary C/E 22. Heat transfersthrough the heat exchanger of the C/E 22 from the HE coolant to thesecondary coolant, heating the secondary coolant and cooling the HEcoolant. The processor 13 controls the flow of the secondary coolant tomaintain desired pressure and temperature of the HE coolant as cooled bythe secondary coolant. The heated secondary coolant then is recycledthrough the ice tanks 188 for further cooling. This continues untileither the primary C/E 20 becomes operational, the secondary C/E 22becomes operational, cooling of the secondary coolant is not desired(e.g., the IT air does not need cooling), or the ice in the tanks 188 isdepleted such that the secondary coolant can no longer adequately coolthe HE coolant. This is indicated by the return of the process 200 tostage 202 (FIG. 6).

Referring to FIG. 11, with further reference to FIGS. 1,3-4, and 12,with particular reference to FIG. 12, a process 290 for adding a heatexchanger module 150 to the HE module section 18 includes the stagesshown. The process 290 is exemplary and not limiting. The process 290may be modified, e.g., by adding, removing, or rearranging stages.

At stage 292, the new HE module 150 is connected to the liquid line 110,recover/vacuum line 132, and the vapor line 134 through flexible tubes.The flexible nature of the tubes helps make connection of the module 150quick and easy. The shutoff valves 152, 154, 156 are all in, or put in,the off position, inhibiting flow of the HE coolant into the module 150.

At stage 294, gas is evacuated from the evaporation tubes 170 of the newHE module 150. The shutoff valve 154 corresponding to the new module 150is opened, allowing gas in the evaporation tubes 170 of the new module150 to be drawn out. The vacuum pump 126 is actuated by the processor 13to suck gas from the tubes 170 through the reclaim/vacuum line 132. Thegas vacuumed by the pump 126 is exhausted to the atmosphere, with thevalve 130 open and the valve 128 closed. The vacuum pump 126 is operateduntil a desired pressure is reached in the tubes 170, e.g., a vacuum of28″ Hg as indicated by the pressure sensor 166. The shutoff valve 154 isclosed and the pump 126 is deactivated.

At stage 296, the new module 150 is put in fluid communication with theHE coolant inlet line 110 for receiving HE coolant. The shutoff valves152 and 156 are opened in response to control signals from the processor13, allowing the HE coolant to flow through the new module 150.

Referring to FIG. 13, with further reference to FIGS. 1-4, and 12, aprocess 300 for removing a heat exchanger module 150 from the HE modulesection 18 includes the stages shown. The process 300 is exemplary andnot limiting. The process 300 may be modified, e.g., by adding,removing, or rearranging stages.

At stage 302, fluid communication between the module 150 and the HEcoolant inlet line 110 is cut off to inhibit the module 150 fromreceiving HE coolant. The shutoff valves 152 and 156 are closed inresponse to control signals from the processor 13, inhibiting the HEcoolant from flowing into the module 150 to be removed.

At stage 304, HE coolant is reclaimed from the evaporation tubes 170 ofthe HE module 150 to be removed. The reclaim/shutoff valve 154corresponding to the module 150 is opened, allowing HE coolant remainingin the evaporation tubes 170 to be drawn out. The processor 13 actuatesthe reclaim pump 126 to pull the HE coolant from the tubes 170 throughthe reclaim/vacuum line 132. The HE coolant recovered by the pump 126 issupplied through the valve 128 (with the valve 130 closed) to the HEcooling receiver 90. The reclaim/recovery pump 126 is operated until adesired pressure is reached in the tubes 170, e.g., 20″ Hg as indicatedby the pressure sensor 166. The shutoff valve 154 is closed and the pump126 is deactivated.

At stage 306, the HE module 150 is disconnected from the flexible tubesconnecting the module 150 to the lines 110, 132, 134. The shutoff valves152, 154, 156 are all in, or put in, the off position, inhibiting flowto/from the lines 110, 132, 134.

Referring to FIGS. 1, 4, and 14, a preferred physical layout of thesystem 10 is shown in a room 400. The system is shown in use in FIG. 14with two rows of equipment racks 312. As shown, some of the racks 312contain power distribution units (PDUs), some contain uninterruptiblepower supplies (UPSs), and others contain other equipment such astelecommunications equipment. The heat exchanger modules 150 havehousings that are connected to legs 314 that extend away from themodules 150 and that include feet 316 that are configured to rest on topof and be attached to the racks 312. The modules 150 are disposed abovethe tops of the racks 312 and directly over a hot aisle 318 between therows of the racks 312. The hot aisle 318 is the region into which theracks 312 exhaust their heated air. The modules 150 are preferablydisposed at least partially, and more preferably substantially entirely,directly over the aisle 318 (as shown, entirely over the aisle 318) withthe modules 150, legs 314, and feet 316 straddling the aisle 318. Themodules 150 are connected to the primary cooling modules 12 and coolantdistribution section 16 through a line 320 that includes the input line110, the output line 134, the reclaim/vacuum line 132 (FIG. 3), andappropriate electrical lines. The modules 12 and the section 16 arepreferably contained in a housing 322 and are preferably disposedremotely from the racks 312, although any of these components could bedisposed in one or more rows of the racks 312. The ice tanks 188 arecontained in a housing 324 disposed remotely from both the racks 312 andthe housing 322. As indicated, some ice tanks 188 can be disposed in aseparate housing 326 that is displaced from the housing 324. The tanks188 may also be disposed in one or more rows of the racks 312. Physicallayouts of the system 10 other than that shown in FIG. 14 are possible.

Other embodiments are within the scope and spirit of the appendedclaims. For example, due to the nature of software, functions describedabove can be implemented using software, hardware, firmware, hardwiring,or combinations of any of these. Features implementing functions mayalso be physically located at various positions, including beingdistributed such that portions of functions are implemented at differentphysical locations. Further, while only one vacuum/reclaim line 134 isshown and described above, multiple lines may be used. For example,separate lines may be used for vacuuming gas and reclaiming HE coolant.In this case, each line would preferably have a shutoff valve to inhibitundesired flow of gas or coolant.

Still other embodiments are within the scope of the invention. Forexample, the cooling module 12 may be implemented without the primaryC/E 20, the compressor 24, or the condenser 26. In this case, thesecondary C/E 22 would preferably be connected to a building chilledwater supply via the lines 72, 74. In this configuration, the coolingmodules 12 could be cascaded as with the modules 12 including theprimary C/E 20, the compressor 24, and the condenser 26. Ice backupcould be used with this arrangement, e.g., if switches/valves wereplaced in the lines 72, 74 that could selectively fluidly couple thebuilding chilled water or the ice storage to the C/E 22. Furtherembodiments could use the cooling modules 12 without the secondary C/E22. Also, the heat exchanger module section 18 may include more than oneheat exchanger, e.g., five or more heat exchangers, and the heatexchanger(s) may have any of a variety of form factors.

Further, while the description above refers to the invention, thedescription may include more than one invention.

What is claimed is:
 1. A system for cooling gas heated by heat-producingelectronic equipment, the system comprising: a heat exchanger configuredto transfer heat from the heated gas to a first coolant; a first coolingmodule connected for fluid communication with the heat exchanger andincluding a first condenser configured to cool and condense incomingfirst coolant from vapor to liquid, the first cooling module beingconfigured to transfer heat from the first coolant to a second coolantto cool the first coolant; a second cooling module connected for fluidcommunication with the heat exchanger and including a second condenserconfigured to cool and condense incoming first coolant from vapor toliquid, the second cooling module being configured to transfer heat fromthe first coolant to a third coolant to cool the first coolant; and acondenser-charge controller configured to regulate a first coolantliquid level in the first condenser.
 2. The system of claim 1 whereinthe condenser-charge controller comprises first and secondcondenser-charge controller subsystems connected and configured tocontrol liquid levels in the first and second condensers, respectively.3. The system of claim 2 wherein the first and second subsystems eachinclude a liquid level sensor configured to determine a liquid level inthe respective condenser, a pump, and a controller coupled to the pumpand the liquid level sensor and configured to regulate the pump toaffect the corresponding liquid level.
 4. The system of claim 3 whereinthe liquid level sensor is a pressure differential sensor and whereinthe cooling module includes a coolant container connected to thecondenser and the pump and the liquid level sensor is connected to thecoolant container to determine the liquid level in the coolantcontainer, the liquid level in the coolant container being related tothe liquid level of the condenser.
 5. The system of claim 3 wherein thefirst and second cooling modules further comprise: a container connectedto the condenser and configured to store the first coolant; a pumpconnected to the container and configured to pump the first coolant fromthe container; a purge mechanism connected to the pump; a purgecontroller coupled to the purge mechanism and configured to actuate thepurge mechanism to purge at least some of the first coolant pumped bythe pump; a fill mechanism connected to the container; and a fillcontroller coupled to the fill mechanism and configured to actuate thefill mechanism to supply liquid first coolant to the container.
 6. Thesystem of claim 5 wherein the purge controller is configured to actuatethe purge mechanism if the pump is operating at about full capacity andthe liquid level of the container rises above an upper threshold leveland/or more than a first threshold amount.
 7. The system of claim 5wherein the fill controller is configured to actuate the fill mechanismif the pump is operating at about minimum capacity and the liquid levelof the container drops below a lower threshold level and/or more than asecond threshold amount.
 8. A system for cooling gas heated by passingthe gas over heat-producing equipment to cool the equipment, the systemcomprising: a heat exchanger including a heat exchanger heat transfermechanism configured to transfer heat from the heated gas to a heatexchanger coolant; a first condensing module connected for fluidcommunication with the heat exchanger and including a first heattransfer mechanism, the first condensing module being configured totransfer heat through the first heat transfer mechanism from the heatexchanger coolant to a first coolant in the first heat transfermechanism; a second condensing module connected for fluid communicationwith the heat exchanger and including a second heat transfer mechanism,the second condensing module being configured to transfer heat throughthe second heat transfer mechanism from the heat exchanger coolant to asecond coolant in the second heat transfer mechanism; and a processorcoupled to the first and second condensing modules and configured toassign a cooling task to each of the condensing modules based uponexpected cooling demand for the heat exchanger coolant and coolingcapacities providable by at least one of the first and or secondcondensing modules; wherein the processor is configured to assign thefirst condensing module as a primary module for cooling the heatexchanger coolant, to assign the second condensing module as a lagmodule, for cooling the heat exchanger coolant, if the expected coolingdemand exceeds a cooling capacity providable by the first condensingmodule, and to assign the second condensing module as a redundant moduleif the cooling capacity of the first condensing module plus the coolingcapacity of any lag condensing modules is at least as great as theexpected cooling demand, the redundant module being designated for usein cooling a backup refrigerant if the primary module and any lagmodules are operational and for cooling the heat exchanger coolant ifthe primary module or any lag modules are not operational.
 9. The systemof claim 8 wherein the first and second condensing modules are coupledin parallel through a single coolant loop to the heat exchanger.
 10. Thesystem of claim 8 wherein the primary condensing module is used to coolthe heat exchanger coolant unless the primary condensing module isinoperative, the lag module, if any, is used to cool the heat exchangercoolant if the cooling demand exceeds the cooling capacity of theprimary module and any other lag module, and the redundant module, ifany, is used to cool the heat exchanger coolant if any of the primaryand lag, if any, modules is inoperative and the cooling demand exceedsthe cooling capacity of the operative primary and lag, if any, modules,and is used to produce ice if the cooling capacity of the operativeprimary and lag, if any, modules at least meets the cooling demand. 11.The system of claim 8 wherein the first and second condensing moduleseach have a cooling capacity that is no greater than an expected coolingdemand for the heat exchanger coolant.
 12. The system of claim 8 whereinthe first and second condensing modules each have a cooling capacitythat is at least as great as an expected cooling demand for the heatexchanger coolant.
 13. The system of claim 8 wherein the second heattransfer mechanism is further configured to transfer heat through thesecond heat transfer mechanism from the heat exchanger coolant to athird coolant in the second heat transfer mechanism.
 14. A system forcooling gas heated by passing the gas over heat-producing equipment tocool the equipment, the system comprising: a heat exchanger including aheat exchanger heat transfer mechanism configured to transfer heat fromthe heated gas to a heat exchanger coolant; a first condensing moduleconnected for fluid communication with the heat exchanger and includinga first heat transfer mechanism, the first condensing module beingconfigured to transfer heat through the first heat transfer mechanismfrom the heat exchanger coolant to a first coolant in the first heattransfer mechanism; and a second condensing module connected for fluidcommunication with the heat exchanger and including a second heattransfer mechanism, the second condensing module being configured totransfer heat through the second heat transfer mechanism from the heatexchanger coolant to a second coolant in the second heat transfermechanism; wherein the system is configured to use excess coolingcapacity of the second condensing module to produce ice for use incooling the heat exchanger coolant if the first condensing module isinoperative.
 15. The system of claim 14 wherein the second heat transfermechanism is further configured to transfer heat through the second heattransfer mechanism from the heat exchanger coolant to a third coolant inthe second heat transfer mechanism.
 16. A system for cooling gas heatedby passing the gas over heat-producing equipment to cool the equipment,the system comprising: a heat exchanger including a heat exchanger heattransfer mechanism configured to transfer heat from the heated gas to aheat exchanger coolant; a first condensing module connected for fluidcommunication with the heat exchanger and including a first heattransfer mechanism, the first condensing module being configured totransfer heat through the first heat transfer mechanism from the heatexchanger coolant to a first coolant in the first heat transfermechanism; a second condensing module connected for fluid communicationwith the heat exchanger and including a second heat transfer mechanism,the second condensing module being configured to transfer heat throughthe second heat transfer mechanism from the heat exchanger coolant to asecond coolant in the second heat transfer mechanism, wherein the secondheat transfer mechanism is further configured to transfer heat throughthe second heat transfer mechanism from the heat exchanger coolant to athird coolant in the second heat transfer mechanism, wherein the secondheat transfer mechanism is further configured to transfer heat throughthe second heat transfer mechanism from the heat exchanger coolant to athird coolant in the second heat transfer mechanism; and an ice storagetank connected to the second heat transfer mechanism, wherein the secondcondensing module includes a battery and a pump that is connected to thebattery and the ice storage tank, and wherein the battery is configuredto power the pump and the pump is connected and configured to circulatethe third coolant between the second heat transfer mechanism and the icestorage tank to cool the third coolant with the ice and to cool the heatexchanger coolant with the third coolant.
 17. The system of claim 16further comprising a processor coupled to the first and secondcondensing modules and configured to assign a cooling task to each ofthe condensing modules based upon expected cooling demand for the heatexchanger coolant and cooling capacities providable by at least thefirst and second condensing modules.
 18. The system of claim 16 whereinthe second heat transfer mechanism is further configured to transferheat through the second heat transfer mechanism from the heat exchangercoolant to a third coolant in the second heat transfer mechanism. 19.The system of claim 13 14 further comprising a processor coupled to thefirst and second condensing modules and configured to assign a coolingtask to each of the condensing modules based upon expected coolingdemand for the heat exchanger coolant and cooling capacities providableby at least the first and second condensing modules.
 20. The system ofclaim 1 further comprising: a coolant distribution subsystem connectedto the heat exchanger and the first and second cooling modules andconfigured to transfer the cooled first coolant from the first andsecond cooling modules to the heat exchanger and to transfer the heatedfirst coolant from the heat exchanger to the first and second coolingmodules; and at least one processor coupled to the coolant distributionsubsystem and the heat exchanger, and configured to: determine a dewpoint of the gas associated with the heat exchanger; monitor a physicalcharacteristic of the first coolant relevant to saturation of the firstcoolant; and control supply of the first coolant to the heat exchangersuch that a combination of temperature and pressure of the first coolantentering the heat exchanger put the first coolant at a saturation pointof the first coolant with a first coolant temperature being above thedetermined dew point temperature.
 21. The system of claim 20 wherein thecoolant distribution subsystem further comprises a first coolanttemperature sensor and a first coolant pressure sensor configured tomonitor temperature and pressure of the first coolant exiting the firstand second cooling modules, the at least one processor being coupled tothe first coolant temperature sensor and the first coolant pressuresensor and configured to regulate the first and second cooling modulessuch that the temperature and pressure of the first coolant exiting thefirst and second cooling modules are at desired levels.
 22. The systemof claim 20 wherein the heat exchanger is part of a heat exchangersection that includes multiple heat exchangers and wherein the at leastone processor is configured to: determine respective dew points of thegas in respective vicinities of each of the multiple heat exchangers;monitor the physical characteristic of the first coolant near anentrance to each of the multiple heat exchangers; and control supply ofthe first coolant to the multiple heat exchangers such that combinationsof temperature and pressure of the first coolant entering respectiveones of the multiple heat exchangers put the first coolant at saturationpoints of the first coolant with respective first coolant temperaturesbeing above respective determined dew point temperatures.
 23. The systemof claim 20 further comprising a heat exchanger temperature (HET) sensorand a heat exchanger humidity (HEH) sensor configured to monitor atemperature and a humidity of the heated gas disposed adjacent to theheat exchanger, the at least one processor being coupled to the HETsensor and to the HEH sensor and being configured to use temperature andhumidity indicia from the HET and HEH sensors to determine the at leastone dew point.
 24. The system of claim 1 further comprising: a firstshutoff valve connected to an output of the first cooling module forreceiving the first coolant and connected to an input of the heatexchanger to selectively permit flow of the first coolant from the firstcooling module to the heat exchanger; a second shutoff valve connectedto an output of the heat exchanger and connected to an input of thefirst cooling module to selectively permit flow of the first coolantfrom the heat exchanger toward the first cooling module; a third shutoffvalve coupled to the heat exchanger; and a pump arrangement connected tothe third shutoff valve and configured to draw gas and the first coolantthrough the third shutoff valve from the heat exchanger.
 25. The systemof claim 24 wherein the pump arrangement is configured to vent the drawngas to a region external to the system.
 26. The system of claim 24wherein the pump arrangement is further connected to the first coolingmodule and is configured to convey the drawn first coolant to the firstcooling module.
 27. The system of claim 24 further comprising aplurality of sets of first, second, and third shutoff valves connectedto the output of the first cooling module, the input of the firstcooling module, and the pump arrangement, respectively, the systemfurther comprising a processor coupled to the first, second, and thirdshutoff valves of each of the plurality of sets and to the pumparrangement, the processor being configured to control the shutoffvalves and the pump arrangement such that: in response to an indicationto add a new heat exchanger to the system with the new heat exchangerbeing coupled to a new set of the first, second, and third shutoffvalves, the processor will cause the first and second shutoff valves ofthe new set to be closed, the third shutoff valve of the new set to beopened, the pump arrangement to draw gas from the new heat exchangeruntil a desired pressure is attained in the new heat exchanger and thenthe processor will cause the third shutoff valve of the new set to beclosed, and the first and second shutoff valves of the new set to beopened; and in response to an indication to remove a certain heatexchanger from the system with the certain heat exchanger being coupledto a certain set of the first, second, and third shutoff valves, theprocessor will cause the first and second shutoff valves of the certainset to be closed, the third shutoff valve of the certain set to beopened, the pump arrangement to draw the first coolant from the certainheat exchanger until a desired pressure is attained in the certain heatexchanger and then the processor will cause the third shutoff valve ofthe certain set to be closed, and the first and second shutoff valves ofthe certain set to be opened.
 28. The system of claim 16 furthercomprising: an outgoing pump coupled to the second condensing module andthe heat exchanger and configured to pump the heat exchanger coolantfrom the second condensing module to the heat exchanger; and wherein thesecond heat transfer mechanism comprises cooling means for transferringthe heat from the heat exchanger coolant through at least one of aplurality of heat transfer elements into at least one of the secondcoolant or the third coolant.
 29. The system of claim 28 wherein thecooling means is configured to select between which one of the secondcoolant and the third coolant to use to cool the first coolant.
 30. Thesystem of claim 28 wherein the cooling means includes primary coolingmeans for cooling the second coolant.
 31. The system of claim 30 whereinthe primary cooling means is configured to cool the third coolant, andthe cooling means is configured to direct the third coolant cooled bythe primary means to the ice storage tank to freeze water stored in theice storage tank.
 32. The system of claim 30 wherein the cooling meansis configured to regulate amounts of the second coolant provided to thesecond condensing module to control at least a temperature of the firstcoolant pumped from the second condensing module.
 33. The system ofclaim 30 further comprising pump regulator means for regulating theoutgoing pump to control pressure of the first coolant such that thefirst coolant entering the heat exchanger will be at saturation while atemperature of the first coolant entering the heat exchanger is above adew point temperature of the heated gas in a vicinity of the heatexchanger.
 34. The system of claim 8 further comprising: a plurality ofequipment racks configured to house the heat-producing equipment, theracks being arranged in rows such that equipment disposed in the rackswill vent hot air into an aisle defined between the rows of racks; and aheat exchanger unit including the heat exchanger, the heat exchangerunit including: a housing configured to contain the heat exchanger; anda mounting apparatus connected to the housing and to at least one rack,the mounting apparatus configured such that the heat exchanger isdisposed at least partially vertically aligned with the aisle.
 35. Thesystem of claim 34 wherein the mounting apparatus is configured suchthat the heat exchanger is disposed at least partially directly over theaisle.
 36. The system of claim 35 wherein the mounting apparatus isconfigured such that the heat exchanger is disposed substantiallyentirely directly over the aisle.
 37. The system of claim 36 wherein themounting apparatus is configured to connect to at least one rack in eachof two different rows of the equipment racks such that the heatexchanger unit straddles the aisle.